Workload mapper for potential problem areas using modules and defect data

ABSTRACT

Embodiments are directed to methods for improving the efficiency at which problem areas are identified and prioritized for an existing large, multi-module software system. In some embodiments, a workload mapper generates workload maps that identify the intersection between defect risk scores accumulated for various modules and a log of the modules that are accesses by a given workload. A graphical user interface (GUI) provides the ability to sort, search, compare and display the workload maps against various sort, search and/or compare criteria.

CLAIM OF PRIORITY UNDER 35 U.S.C. §120

The present application is a continuation of U.S. patent applicationSer. No. 14/193,078, titled “Z/OS WORKLOAD MAPPER FOR POTENTIAL PROBLEMAREAS USING MODULES AND DEFECT DATA,” filed Feb. 28, 2014, which isincorporated herein by reference in its entirety.

BACKGROUND

The present disclosure relates generally to identifying problem areas ofa software system, and more specifically to improving the efficiency ofsystems used to identify and prioritize potential and actual problemareas of large, multi-module software systems.

Mainframe computers process large amounts of data, such as censusinformation, industry/consumer statistics and financial transactions. Inaddition to processing speed, effective mainframe computers provideinternal design redundancy, extensive throughput capabilities andbackward compatibility with older software. z/OS® is a 64-bit operatingsystem for mainframe computers manufactured and sold by InternationalBusiness Machines® (IBM®). z/OS offers the attributes of modernoperating systems, along with backward compatibility to functionalitythat originated in the 1960s and subsequent decades.

Under the high demands of current computing environments, mainframeusers are implementing increasingly more sophisticated configurationsand systems. Mainframe users often tailor their operating system for avariety of performance benefits and space enhancements, which canunintentionally increase the number and frequency of potential softwarecode problem areas. Also, operating systems that have several decades inthe marketplace, and that prioritize backward compatibility, becomeincreasingly large and complex with each new version of the system. Forexample, z/OS after several decades in the marketplace now includesseveral tens of thousands of computer libraries/modules of code, andeach new z/OS release adds additional libraries/modules that furtherincrease the number and frequency of potential software code problemareas.

Vendors routinely test software to identify problem areas. For example,IBM uses so-called “test case” programs to test z/OS. Test cases aresmall programs that are run on the operating system to verify quality.As previously noted, as a software system grows in size and complexity,the number of test cases associated with that software system grows aswell.

Problem areas are also identified during usage. If an end userencounters a known problem area, the vendor's product support functioncan direct the user through steps to correct the problem area, such asapplying a new software patch. If an end user encounters a new,unidentified problem area, the issue is typically catalogued andverified, then a solution (e.g., a software patch) is generated andimplemented as need.

Thus, identifying and prioritizing problem areas can be a challenge forlarge software systems having unique user configurations, several tensof thousands of modules, several tens of thousands of test cases,backward compatibility and many decades in the marketplace. Such systemscan require a significant investment of time and resources to run everyapplicable test case for every new release of the system. Accordingly,there is a need for devices, systems and methods to improve theefficiency of systems that identify and prioritize potential and actualproblem areas of large, multi-module, multi-test case software systems.

BRIEF SUMMARY

Embodiments are directed to a method of identifying potential problemareas of a system comprising a plurality of workloads comprising aplurality of modules, the method comprising: executing at least one ofthe plurality of workloads; capturing a log of execution modules thatare accessed during said executing; developing a risk score databasecomprising risk scores for at least some of the plurality of modules;and generating workload maps comprising intersections between said logof execution modules and said risk scores for at least some of theplurality of modules; wherein the potential problem areas comprise saidintersections.

Additional features and advantages are realized through the techniquesdescribed herein. Other embodiments and aspects are described in detailherein. For a better understanding, refer to the description and to thedrawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The subject matter which is regarded as the present disclosure isparticularly pointed out and distinctly claimed in the claims at theconclusion of the specification. The foregoing and other features andadvantages are apparent from the following detailed description taken inconjunction with the accompanying drawings in which:

FIG. 1 is a block diagram of a computer system embodying the presentdisclosure;

FIG. 2 is a block diagram illustrating additional details of a computersystem embodying the present disclosure;

FIG. 3 is a flow diagram illustrating a methodology embodying thepresent disclosure;

FIG. 4 illustrates a display configuration for a graphical userinterface (GUI) embodying the present disclosure;

FIG. 5 illustrates another display configuration for a GUI embodying thepresent disclosure; and

FIG. 6 illustrates another display configuration for a GUI embodying thepresent disclosure.

In the accompanying figures and following detailed description of thedisclosed embodiments, the various elements illustrated in the figuresare provided with three digit reference numbers. The leftmost digit ofeach reference number corresponds to the figure in which its element isfirst illustrated.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the relevant arts, the presentdisclosure may be embodied as a method, system, or computer programproduct. Accordingly, embodiments of the present disclosure may beimplemented entirely in hardware, entirely in software (includingfirmware, resident software, micro-code, etc.) or in an embodimentcombining software and hardware. The various embodiments may allgenerally be referred to herein as a “circuit,” “module,” or “system.”Furthermore, the present disclosure may take the form of a computerprogram product on a computer-usable storage medium havingcomputer-usable program code embodied in the medium.

Any suitable computer usable or computer readable medium may beutilized. The computer-usable or computer-readable medium may be, forexample, but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. More specific examples (a non-exhaustive list) ofthe computer-readable medium would include the following: an electricalconnection having one or more wires, a portable computer diskette, ahard disk, a random access memory (RAM), a read-only memory (ROM), anerasable programmable read-only memory (EPROM or Flash memory), anoptical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a transmission media such as those supportingthe Internet or an intranet, or a magnetic storage device. Note that thecomputer-usable or computer-readable medium could even be paper oranother suitable medium upon which the program is printed, as theprogram can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this disclosure, acomputer-usable or computer-readable medium may be any medium that cancontain, store, communicate, propagate, or transport the program for useby or in connection with the instruction execution system, apparatus, ordevice.

Many of the functional units described in this specification have beenlabeled as modules. The present disclosure applies to a wide variety ofmodule implementations. For example, a module may be implemented as ahardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices or the like.

Modules may also be implemented in software for execution by varioustypes of processors. An identified module of executable code may, forinstance, comprise one or more physical or logical blocks of computerinstructions which may, for instance, be organized as an object,procedure, or function. Nevertheless, the executables of an identifiedmodule need not be physically located together, but may comprisedisparate instructions stored in different locations which, when joinedlogically together, comprise the module and achieve the stated purposefor the module.

A module of executable code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data may be identified and illustrated hereinwithin modules, and may be embodied in any suitable form and organizedwithin any suitable type of data structure. The operational data may becollected as a single data set, or may be distributed over differentlocations including over different storage devices.

Computer program code for carrying out operations of the presentdisclosure may be written in an object oriented programming languagesuch as Java™, Smalltalk, C++ or the like (Java and all Java-basedtrademarks and logos are trademarks of Sun Microsystems, Inc. in theUnited States, other countries, or both). However, the computer programcode for carrying out operations of the present disclosure may also bewritten in conventional procedural programming languages, such as the“C” programming language or similar programming languages. The programcode may execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Embodiments of the present disclosure are described below with referenceto flowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thepresent disclosure. It is noted that various connections are set forthbetween elements in the present disclosure and in the drawings. Theseconnections, unless specified otherwise, may be direct or indirect, andthe present disclosure is not intended to be limiting in this respect.Accordingly, a coupling of entities may refer to either a direct or anindirect connection. Further, it will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. The computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks. The computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meansthat implement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer or other programmable data processing apparatusto cause a series of operational steps to be performed on the computeror other programmable apparatus to produce a computer implementedprocess such that the instructions which execute on the computer orother programmable apparatus provide steps for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

FIG. 1 is a schematic block diagram of a system 100 embodying thepresent disclosure. System 100 includes a computer 106, a processor 108,a memory 110, a user interface (UI) 104 and a display 102, configuredand arranged as shown. Memory 110 includes an operating system 114,which, in one embodiment, is a z/OS operating system. Under the presentdisclosure, operating system 114 includes a workload mapper 116, aworkload analyzer 118, a plurality of workloads/test cases 122, and aplurality of modules 124, 126, 128, configured and arranged as shown.

FIG. 2 illustrates additional details of system 100, along withadditional details of certain functional relationships. It should againbe emphasized here that the functional relationships shown in FIG. 2 donot imply a particular sequence of events or flow ofinformation/signals. For instance, the actions may be performed in adiffering order or actions may be added, deleted or modified. Also, theterm “coupled” describes having a signal path between two elements anddoes not imply a direct connection between the elements with nointervening elements/connections therebetween. All of these variationsare considered a part of the present disclosure. As shown in FIG. 2,workloads/test cases 122 are coupled to workload analyzer 118, which iscoupled to workload mapper 116. A defect database 204 and a set of riskcriteria 206 are coupled to a risk score database 202, which is coupledto workload mapper 116. Workload mapper 116 is coupled to a graphicaluser interface (GUI) section 105 of UI 104, which is in turn coupled todisplay 102. Additional details of FIG. 2 are described later in thisdisclosure.

Computer 106 of FIG. 1 is typically a mainframe computer, but may alsobe a personal computer, laptop computer, workstation, etc. with amainframe operating system or a mainframe operating system emulator.Processor 108 may be a single processor, multiple processors, a chipsetwith a processor, or any other configuration of electronic parts andcomponents comprising a processor that may process lines of computercode. Memory 110 may be random access memory (“RAM”), a hard disk drive,tape storage, an optical drive, flash memory, or other memory capable ofstoring data. Memory 110 may be connected to computer 106 over acomputer network such as a LAN, WAN, SAN, etc. Memory 110 may also beconnected to computer 106 over a communication bus such as a SmallComputer Systems Interface (“SCSI”), Advanced Technology Attachment(“ATA”), or other appropriate communication bus.

UI 104 may be a terminal, a personal computer, a laptop computer, atablet computer, a personal digital assistant (“PDA”) or similarcomputing device capable of providing a way for a user to input data tocomputer 106. In one embodiment, UI 104 includes a keyboard connected tocomputer 106. UI 104 may be connected to computer 106 by direct wiringor through a computer network such as the Internet, a local area network(“LAN”), a wide area network (“WAN”), a wireless network, a storage areanetwork (“SAN”), or any other connection capable of transmitting datafrom UI 104 to computer 106.

Display 102 may be a computer monitor, a liquid crystal display (“LCD”)monitor, or any device capable of displaying data on a screen, paper, orother media. In one embodiment, display 102 displays GUI 105 thatfacilitates the display of data to a user. Examples of displays 102 a,102 b showing a suitable GUI format are shown in FIGS. 4 and 5 anddescribed in more detail later in this disclosure.

Turning now to a description of the disclosed embodiments, and withreference to the configurations shown in FIGS. 1 and 2, the presentdisclosure relates in general to large and complex operating systems.Although the present disclosure is described in connection with a z/OSoperating system, the disclosure is broad enough to cover any large andcomplex software system with multiple modules and multiple associatedtest cases. Defect database 204 shown in FIG. 2 is a database maintainedin the course of operating system 100. In a conventional configuration,defect database 204 is a database of all historical problem areas wherea fix was provided. Often, the defects that populate defect database 204result from defects identified by customers using operating system 114in the marketplace. Using the IBM z/OS software code as an example, whena new, unidentified potential problem is identified, a problemmanagement record (PMR) is opened on a case-by-case basis by IBMsupport. Every PMR receives a unique identification number, and allcommunication and support activities are handled under this number. IBMspecialists attempt to isolate the identified error and detect thefailing database software component (e.g., a module). When the problemis reproducible and isolated, an authorized program analysis report(APAR) is opened. Thus, an APAR is a named issue within an IBM program,and is opened after a customer or IBM support personnel discover aproblem with the database software code. Every APAR has a uniqueidentification number. After the IBM development team solves the problemdefined in the APAR, the database software fixes will be provided tocustomers. High impact pervasive (HIPER) APARs are critical problemareas of which all customers should be aware.

Additional information about the defects identified in defect database204 is provide by various risk criteria 206, including but not limitedto the modules or other operating system artifacts that were updated toaddress the problem areas; the severity of the defect; what userreported the defect (may be more than one); was the fix classified asone for immediate application; was the defect a fix to another defect;prioritize recent defects over older defects; and other optionalweighting mechanisms. Both defect database 204 and various risk criteria206 feed information into risk score database 202, which uses theinformation to determine as risk score for the various modules that wereidentified in defect database 204.

Workloads/test cases 122 generally comprise a variety of modules (122,124, 126 shown in FIG. 1). As previously described, test cases are smallprograms that are run on a computer system (e.g., an operating system)to verify quality. As a software system grows in size and complexity,the number of workloads, test cases and load modules associated withthat software system grows as well. Workload analyzer 118 implement abasic scrubbing technique that analyzes workloads/test cases and logsthe load modules and/or other code paths that are accessed in executingthe workloads/test cases. Workload analyzer 118 provides the logs of themodules or code paths that are accessed in executing the workloads/testcases to workload mapper 116, which maps the logs against the riskscores developed through risk score database 202. Maps 117 developed byworkload mapper 116 are provided to UI 104 and GUI 105, which includesthe ability to search, sort and/or compare maps 117 against a variety ofsearch, sort and/or comparison criteria. Display 102 displays theresults generated by GUI 105.

FIG. 3 illustrates a methodology 300 embodying the present disclosure.It should be emphasized here that the functional relationships shown inFIG. 3 do not imply a particular sequence of events or flow ofinformation/signals. For instance, the actions may be performed in adiffering order or actions may be added, deleted or modified. Also, theterm “coupled” describes having a signal path between two elements anddoes not imply a direct connection between the elements with nointervening elements/connections therebetween. All of these variationsare considered a part of this disclosure. Methodology 300 may beimplemented by processor 108 (shown in FIG. 1) to generate maps 117(shown in FIG. 2). As shown, methodology 300 begins with a block 302 inwhich the operating system is initialized. Methodology 300 then moves toa block 304 to enable instrumentation/tracing, which is performed byworkload analyzer 118 (shown in FIG. 1). Block 306 then runs workloadsand/or test cases 1-M through workload analyzer 118, and block 308disables the instrumentation/tracing. Block 310 generates a tracelog/map file, again via workload analyzer 118. Block 312 updates riskscore database 202 (shown in FIG. 2), and block 314 uploads tracelog/map to UI 104 and GUI 105. Block 316 displays the trace log/mapresults.

Using the IBM z/OS system as an example, FIGS. 4, 5 and 6 are displays102 a, 102 b, 102 c showing how maps 117 (shown in FIG. 1) may besearched, sorted and/or displayed. FIGS. 4, 5 and 6 are exampleconfigurations, and a wide range of configurations are possible underthe present disclosure. The search, sort and/or compare functionality ofUI 104 and GUI 105 provide an easily navigated way to dissect tracelog/map results. FIG. 4 shows a table of workloads/test cases and theirrelated risk score based on the modules that the workload/test caseuses. FIGS. 5 and 6 show the results of a search, sort and comparefunction for various workloads/test cases of interest. Thus, for allworkloads/test cases, as well as component load modules that have beenidentified by workload mapper 116 (shown in FIGS. 1 and 2), displays 102a, 102 b, 102 c may provide a variety of general workload/test case metadata, including but not limited to: the individual who submitted theworkload/test case; when the workload/test case was submitted; a smalldescription of the workload/test case provided by the user; a list ofall load modules that were touched/executed during the workload/testcase execution; and a computed risk score of all load modules.Additionally, for all load modules, including the detailed analysis foreach load module, displays 102 a, 102 b, 102 c may include: the name anddescription; a computed risk score for the load module; and a list ofall workloads that have touched/executed the load module.

Thus, for system 100 having a large number of modules 124, 126, 128 anda large number of workloads/test cases 122, a vendor responsible foridentifying and prioritizing software code problem areas is capable ofefficiently answering a variety of important questions, including butnot limited to: which load modules are executed by a workload/test case;which workloads/test cases will execute a particular load module or codepath; under time constraints, which workloads/test cases should be runfirst in order to validate quality for a specific load module(s); whichload modules are high risk for customers (risk score); which loadmodules are high risk for customers and have no automated workloads thatcan be run to validate the load module; and, under time constraints,where should new test case creation resources be focused (e.g., a loadmodules may have a high risk score but no associated workloads).

Various embodiments of the present disclosure have been described withreference to the related drawings. Alternate embodiments may be devisedwithout departing from the scope of this disclosure. It is noted thatvarious connections are set forth between elements in the presentdisclosure and in the drawings. These connections, unless specifiedotherwise, may be direct or indirect, and the present disclosure is notintended to be limiting in this respect. Accordingly, a coupling ofentities may refer to either a direct or an indirect connection.

In some embodiments, various functions or acts may take place at a givenlocation and/or in connection with the operation of one or moreapparatuses or systems. In some embodiments, a portion of a givenfunction or act may be performed at a first device or location, and theremainder of the function or act may be performed at one or moreadditional devices or locations.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprises” and/or “comprising,”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, element components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thepresent disclosure has been presented for purposes of illustration anddescription, but is not intended to be exhaustive or limited to the formdisclosed. Many modifications and variations will be apparent to thoseof ordinary skill in the art without departing from the scope and spiritof the disclosure. The embodiments were chosen and described in order tobest explain the principles of the disclosure and the practicalapplication, and to enable others of ordinary skill in the art tounderstand the disclosure for various embodiments with variousmodifications as are suited to the particular use contemplated.

The diagrams depicted herein are illustrative. There may be manyvariations to the diagram or the steps (or operations) described thereinwithout departing from the spirit of the disclosure. For instance, theactions may be performed in a differing order or actions may be added,deleted or modified. Also, the term “coupled” describes having a signalpath between two elements and does not imply a direct connection betweenthe elements with no intervening elements/connections therebetween. Allof these variations are considered a part of the present disclosure.

What is claimed is:
 1. A computer implemented method of analyzing asystem comprising a plurality of workloads comprising a plurality ofsoftware system modules, the method comprising: storing historicaldefects of ones of the plurality of software system modules for whichdefect fixes have been provided; storing risk criteria of the historicaldefects and the defect fixes; executing at least one of a plurality oftest cases comprising test case software that, when run on the softwaresystem, tests the software system; capturing a log of test caseexecution modules comprising software system modules that are accessedduring said executing; developing, by a processor, a risk score databasecomprising risk scores for the plurality of software system modules forwhich defect fixes have been provided, wherein the risk scores are basedat least in part on the risk criteria; generating, by said processor,workload maps comprising intersections between said log of test caseexecution modules and said risk scores for at least some of theplurality of software system modules; and based at least in part on saidworkload maps, generating data that identifies a prioritization of theplurality of test cases.
 2. The computer implemented method of claim 1further comprising: incorporating said workload maps into a graphicaluser interface (GUI); and displaying said GUI on a display.
 3. Thecomputer implemented method of claim 2 wherein said GUI comprises anability to search said workload maps based on at least one searchcriterion.
 4. The computer implemented method of claim 3 wherein saidGUI comprises an ability to sort said workload maps based on at leastone sort criterion.
 5. The computer implemented method of claim 4wherein said GUI comprises an ability to compare said workload mapsbased on at least one comparison criterion.
 6. The computer implementedmethod of claim 1 wherein said risk scores are based at least in part ondefect data from a defect database.
 7. The computer implemented methodof claim 6 wherein said at least one risk criterion comprises at leastone of: a weighted risk criterion; a severity assessment of said defectdata; and a frequency of occurrence of said defect data.
 8. The computerimplemented method of claim 1 wherein the system comprises an operatingsystem.
 9. A computer program product for analyzing a software systemcomprising a plurality of workloads and a plurality of software systemmodules, the computer program product comprising a computer usablemedium having a computer readable program, wherein the computer readableprogram, when executed on a computer, causes the computer to perform amethod comprising: storing historical defects of ones of the pluralityof software system modules for which defect fixes have been provided;storing risk criteria of the historical defects and the defect fixes;executing at least one of a plurality of test cases comprising test casesoftware that, when run on the software system, tests the softwaresystem; capturing a log of test case execution modules comprisingsoftware system modules that are accessed during said executing;developing, by a processor, a risk score database comprising risk scoresfor the plurality of software system modules for which defect fixes havebeen provided, wherein the risk scores are based at least in part on therisk criteria; generating, by said processor, workload maps comprisingintersections between said log of test case execution modules and saidrisk scores for at least some of the plurality of software modules; andbased at least in part on said workload maps, generating data thatidentifies a prioritization of the plurality of test cases.
 10. Thecomputer program product of claim 9 further comprising: incorporatingsaid workload maps into a graphical user interface (GUI); and displayingsaid GUI on a display.
 11. The computer program product of claim 10wherein said GUI has an ability to search said workload maps based on atleast one search criterion.
 12. The computer program product of claim 10wherein said GUI has an ability to sort said workload maps based on atleast one sort criterion.
 13. The computer program product of claim 10wherein said GUI has an ability to compare said workload maps based onat least one comparison criterion.